NON-LOCALLY-FREE LOCUS OF O'GRADY'S TEN DIMENSIONAL 

EXAMPLE 

YASUNARI NAGAI 

Abstract. We give a completely explicit description of the fibers of the natu- 
ral birational morphism from O'Grady's ten dimensional singular moduli space 
of sheaves on a K3 surface to the corresponding Donaldson-Uhlenbeck com- 
pactification. 
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i — ,■ Introduction 

o, 

O'Grady's construction of a new example of irreducible symplectic manifold 
HO'G991 had a big impact to the study of holomorphic symplectic manifold, as 
it has been understood to be difficult to construct an irreducible symplectic man- 
ifold that is not deformation equivalent to the examples given in [Bea83]. Since 
then, several mathematicians studied the properties of O'Grady's example. In 
,-j- ■ the course of the study, it turned out to be important to understand the locus of 

non-locally-free sheaves on the singular moduli space that O' Grady considered, 
see e.g. [ Rap08[|Per091l . The locus is closely related to the topology, the sin- 
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gularities, and the geometry of O'Grady's example. Actually, O'Grady already 
considered the non-locally-free locus of the moduli space in his article HO'G99l 
to show that the example he constructed has the different second Betti number 
than that of previously known examples. In this article, we obtain an explicit and 
more or less complete understanding of the non-locally-free locus of O'Grady's 
singular moduli space. 

Let 5 be a projective K3 surface with Picard number one. What we mean by 
"O'Grady's singular moduli space" is the moduli space M of Gieseker-semistable 
sheaves on S of rank 2, c\ = 0, and C2 = 4. By the general theory ( HHL97H , 
Definition-Theorem 8.2.8), we have a projective morphism 

(p:M^ M DU 

to the Donaldson-Uhlenbeck compactification M DU . In our case, this turns out 
to be a birational morphism and the exceptional divisor B of <p is nothing but the 
non-locally-free locus of the moduli space M. Our Main Theorem (Theorem ll.il ) 
gives an explicit description of the non-trivial fibers of the morphism <p. 
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Theoretically, one can describe every fiber of the morphism (p as a GIT quo- 
tient of an appropriate sub variety of a Quot scheme (see § 1 , see also op. cit. , 
Chap. 8). However, such a description is not suitable for an explicit study; it is 
almost hopeless to really calculate the associated homogeneous coordinate ring, 
for the defining equation, or the Grothendieck-Pliicker embedding, of a Quot 
scheme can already be very complicated. 

Instead of Quot scheme, we use a quiver-variety-like description of the fiber 
of <p (§2). An obvious merit of this description is that one can really calculate 
the homogeneous invariant ring corresponding to the GIT quotient description 
of the fiber within the framework of classical invariant theory (§3). Here we 
need to appeal for a brute force calculation using Grobner basis to determine the 
relations among the generators of the invariant ring. We should also remark that 
the description cannot be globalized, meaning that one cannot obtain an explicit 
description of the whole non-locally-free locus B by this method. 

Acknowledgment. The author would like to express his gratitude to Manfred 
Lehn for his suggestions and encouragements. He largely benefited from the 
discussions with him. He also thanks Arvid Perego for stimulating discussions. 
Most part of this work was done when the author was in Johannes Gutenberg- 
Universitat Mainz, where he was supported by SFB/TR 45 of DFG, Deutsche 
Forschungsgemeinschaft. 

1. The non-locally-free locus 

Let S be a K3 surface with Pic (5) =Z[H], where H is an ample divisor on S, 
and let M be the moduli space of //-semistable torsion free coherent sheaves on 
S with rank 2, c\ = 0, and C2 = 4. If we denote the locus of strictly semistable 
sheaves by E, one can immediately know that M is singular along E. Moreover, 
we have a stratification E = E° II E 1 by 

e° = {[/ z ©%] \z,wem\b 2 (s),z^w}, 

E^j^JlZGHilb 2 ^)}, 
where the square brackets stand for the S-equivalence classes ( HO'G99L Lemma 
1.1.5). 

Let B be the locus of non-locally free sheaves on M. Obviously, B contains E. 
As is explained in the introduction, B can be captured as the exceptional divisor 
of a projective birational morphism 

(p:M^ M DU 
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to the Donaldson-Uhlenbeck compactification ( HHL971 , Chap. 8). If [E] G B, the 
double dual E** is a jU-semistable sheaf and we have a short exact sequence 

— >E — >E** — ► Q(E) — > 

with Q(E) of length 02(E) — C2 (£"**). We define the associated cycle y(Q) to an 
artinian coherent sheaf Q by 

7(2) = £ length^) ■ p G Sym l (S) (I = length(Q)). 

pes 

The correspondence <p is, roughly speaking, given by 

E^(E**,y(Q(E))) (1) 

( IHL97I . Theorem 8.2.11). In our case, wehave£** = ff® 1 and length ( Q (E)) = 
4 for every [E] G B ( HO'G99H . Proposition 3.1.1). This means that the first factor 
of the correspondence (OQ) is always trivial, so that the restriction of <p to B is of 
the form 

q> ]B :B^Sym 4 (Sy, E^y(Q(E)). 
From this, we get a GIT description of B as 

5^Quot(^® 2 ,4)//5L(2), 

where Quot(^P ,4) stands for the Quot scheme of length 4 quotients of fff 
(cf. HO'G99ll . §3.4). We have a similar description of the fibers of <P| S as follows. 
There is a natural morphism Quot(^ 2 ,4) —$ Sym 4 (5) by the correspondence 

Denote by Quoty the fiber of this morphism over 7 G Sym 4 (5). Then we have 

<p^ l (Y) = Quot Y //SL(2). (2) 

O' Grady proved that the general fiber of <pi s , namely the case where 7 consists of 
four different points on S, is isomorphic to P 1 (op. cit., Proposition 3.0.5). Note 
that from this fact, we know that B is a codimension 1 subvariety of M, since 
dimM = 10. 

Let 7] = [l 711 ^ 772 ^ 3 ^ 774 ] be a partition of 4, namely, 4 = 1 ■ 77 1 +2 ■ T] 2 + 3 • 
7]3 +4 • 774 with r\i non-negative integers. Then, we have a natural stratification 

Sym\S) = \\S^\ 



S (n) 




Pij G S are distinct points 
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A O-cycle 7 G Sym 4 (5) is said to be of type r\ if 7 G S^\ Let 5 r be the fiber of 
<pig : 5 — i- Sym (5) vv/f/z f/?e reduced scheme structure. 

Theorem 1.1. Let 7 G Sym 4 (5), 5 r , an J E as above. 

(i) 7f 7/5 of type [l 4 ], 5 r = P 1 anJ (5yflE) re( / is a three point set. 
(ii) Ifyis of type [l 2 , 2], 5 r = P 2 anJ (5y nE),.^ w a disjoint union of a line I 
and a point P. More precisely, ify=p\+p2 + 2q, 

I = {[Ipi+pz®Iz] I Z G Hilb 2 (S),SuppZ = {a}} = P 1 , 

(iii) Ifyis of type [2 2 ], By is a quadric cone in P 4 . (ByC\H) re d is a disjoint union 
of a smooth hyperplane section T and the vertex P of By. More precisely, if 
y=2 Pl +2 P2 , 

T = {[/ Zl ®/z 2 ] I Zi G Hilb 2 (S),SuppZ = { Pl } (i = 1,2)} = P 1 x P 1 , 
P - \I® 2 1 

(iv) If y is of type [1,3], 5y is isomorphic to a quadric of rank 2 in P 4 , namely, 
the singular scroll in P 4 ?/za? is the image o/Ppi (€?® 2 © ^(2)). (5 r fi E) re ^ 
is f/ze /me / of vertices of By parametrizing 

I = {[Ip+ q ®Iz] I Z G Hilb 2 (5),SuppZ= {a}} £ P 1 

ifr=p+iq- 

(v) 7/7 is of type [4], 5 r z's an irreducible divisor of the singular scroll Ppi ( i^® 3 © 
€^( 1 ) © ^(4) ) C P 9 . 77ze s/rict transform B' of By under the natural proper 
birational morphism 

P P i (^® 3 © 47(1) © tf(4)) -)• P P i (€?® 3 © ^(1) © €?(4)) 

is a topologically locally trivial family of quadric hypersurfaces of rank 
2 in the fiber P 4 of P P i (^® 3 © G{\) © tf (4)). (B y n L) red is the plane 
IT = Ppi (^® 3 ) parametrizing 

n = {[/ Zl ©/z 2 ] I Z G Hilb 2 (5),SuppZ = {p} (i = 1,2)} =* P 2 

if y = 4/7. Moreover, the envelope A 0/ f/?e family of lines on Y\ that is the 
image of the vertex lines of the fibers ofB' is a non-singular quadric on IT 
parametrizing 

A = (BynL% d = {[If] I Z G Hilb 2 (5),Supp(Z) = {p}}. 

The rest of the article is devoted to the proof of this theorem. Note that the 
first assertion (i) is nothing but Proposition 3.0.5 of [0'G99|. 
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Remark 1.1.1. From a view to the geometry of irreducible symplectic mani- 
fold, it should be more interesting to look at the strict transform By of By under 
O'Grady's resolution n:M^M. According to HLS06L O'Grady's symplectic 
resolution % is nothing but the blowing-up along the locus of strictly semistable 
sheaves H rec j. However, it is not so easy to analyze the induced blowing-up 
By = Bl B n r£ red \By — ¥ By, since the scheme theoretic intersection By fl (£ re ^) 
tends to have highly non-trivial non-reduced scheme structure over deeper strata. 

2. GIT DESCRIPTION OF THE FIBER OF (p 

The description © of By by Quot scheme is not suitable to prove Theorem ll.il 
since Quot y already has a complicated projective geometry so that it is hopeless 
to put the GIT quotient involved in © into an actual calculation. Instead, we use 
a quivar-variety-like description of By, which can be seen as an analogy of the 
description of Hilb"(C 2 ) as a hyper- Kahler quotient (see, for example, HNak99l 
Chap. 2 & 3, in particular, Theorem 3.24). 

Definition 2.1. Let V = C 2 . For a O-cycle y = Y, m iPi £ Sym 4 (5), we define a 
sheaf of C- vector spaces £}y of finite length by 

£ x =l £mi ifx = pu 
1 otherwise. 

Define 

N m = {(A,B) E sl(C ra )® 2 I [A,B] = 0, A l B J = O for i + j = m}, 

and define Ny by 

AT y = W? 1 x Nf x Nf x A^ 4 

if yis of type r\ = [I 111 ,2^ 2 , 3 713 ,4^ 4 ]. Here we note that Ni is a reduced one point 
set. We define Y r by 

y r = Hom c (V,r(i2 r ))xAV, 

and define G y = GL(V) x Aut( J? 7 ). We always let Aut(i2 r ) act on N y by the 
adjoint action at each point of the support of J2y. Gy can always be seen as a 
subgroup of GL(V) x GL(C 4 ) by diagonal embedding. We define a character % 
by 

X = (det v r 2 • (det c4 ) : GL(V) x GL(C 4 ) -)■ C* 
and let Xy be the composition 

Xy : Gy -+ GL(V) x GL(C 4 ) -4 C*. 
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Theorem 2.2. Notation as above. We have an isomorphism 

0A(y y ) G ^ 

n=0 

where A(Yy) is the affine coordinate ring ofYy and A(Yy) Y,x v is the vector sub- 
space consisting of Gy-semi-invariants ofA(Yy) whose character is %y- 

Remark 2.2.1. Let 

¥= (y, a) e Y r = Hom(V,r(^ r )) x N r . 

Since N 7 parametrizes ^-module structures on £}y, *P defines a morphism of 
£?s- m °dules 

V <S> @s ->• «%• 
We denote this morphism also by *F slightly abusing the notation. In particular, 
if it is surjective, *P defines a point *F G Quot y C Quot(V ® ^s,4). Moreover, 
every point in the Aut(J2y) -orbit of *F corresponds to *F and we can prove 

(Quot y ) rerf = 7 y // Aut(^ y ). 

det^ r 

Remark 2.2.2. ^ y is the character associated to the determinant line bundle for 
the universal family over the Quot scheme. It will turn out that %y is the only 
meaningful polarization in the calculation of invariant rings in §3. 

Remark 2.2.3. There is nothing to do with the condition C2 = 4 in the descrip- 
tion of the fiber By of the morphism to Donaldson-Uhlenbeck compactification 
in Theorem 12.21 We can easily generalize the theorem to the case of moduli 
space M$(2, 0, 2k) of semistable sheaves of rank 2, c\ = 0, and C2 = 2k for every 
positive integer k. 

By the standard theory of GIT construction of a moduli space, the theorem is 
a consequence of the following 

Proposition 2.3. Notation as above. Let *P = (iff, a) G Yy. Then, the following 
are equivalent: 

(i) The morphism of ffs-modules *P : V<S> &s ~ * &y is surjective and E = Ker \P 

is semistable (resp. stable). 
(ii) *P is surjective and for every one dimensional subspace W CV, dim v I / ( W <8> 

Os) > 2 (resp. dim^W <g> O s ) > 2 ). 
(iii) m is a {Gy,%)-semistable (resp. (Gy,x)-stable) point. 
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Proof. Noting that E = Ker^ is automatically /i-semistable and any destabi- 
lizing subsheaf F of E corresponds to a one dimensional subspace W C V by 
F** = w Cg> @s, the equivalence of (i) and (ii) is nothing but Lemma 1.1.5 of 
K)'G991 . 

The equivalence of (ii) and (iii) is just an application of Hilbert-Mumford's 
numerical criterion. We demonstrate the case where y is of type [1 2 ,2], as the 
proofs in the other cases are similar. A reader can skip this part and go promptly 
to §3, for the argument here is classical and standard. 

(2.3.1) Let Y=P1 +P2 + 1q and B y = Pi © @ P2 @£ m with B yA = C 2 . Then, 

Y y = Hom(V, C) x Hom(V, C) x Hom(V, C 2 ) x N 2 , 

G y = GL(V) xCxCx GL(C 2 ), 

X r = (det v )~ 2 ■ id c * -id c * -(det c2 ). 
For a one parameter subgroup (1-PS, in short) A : C* — > Gy, we define the pairing 
(Xy^) = m by Xy ° h(t) = t m for t EC*. According to a form of the numerical 
criterion ([Kin94|, Proposition 2.5), \P E Yy is (G r , ^y)-semistable if and only 
if (# y , A) ^ holds for every 1-PS A of G y such that the limit lim^oMO ' ^ 
converges. Moreover, a semistable point *P is stable if and only if the strict 
inequality (#y,A) > holds for every non-trivial 1-PS A of Gy such that the 
limit lim^o A (?) • *P converges. 

(2.3.2) Represent a point M* e 7 r by matrices 
^=((x i x 2 Uy 1 y 2 \Z=( Zn Z A;A=( an ai2 V J B=^ 11 J 12 

V ^'^-w \^ 21 Z22 y ^21 a 22y /' V^i ^22 

using bases for V and =2 r . For r = (n, • • • , r 6 ) G Z 6 , we define a 1-PS A r : C* — > 
Gy by 

A r (0 = (diag(^,^),^,r r *,diag(^,^)). 

Then, we have 

a n f^-a l2 \ ( b n f^-bn 

r r * +r *-a 2 i a 22 r[r r 5 +r s-b 2 i b 22 



Note that any 1-PS A is equivalent to A r for some r up to a base change of V and 

CtyA(O) = -2ri-2r 2 + r 3 +r 4 + r 5 + r 6 . 



J3y, and that 
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(2.3.3) First, we prove (iii)=Kii). Assume that *P is not surjective. This is 
equivalent to assume either (a) (xi x%) = or [y\ V2) = 0, or (b) rankZ ^ 1 
and AZ = BZ = O. In the case (a), say, if we have (xi X2) = 0, A r for r = 
(0,0, —1,0,0,0) is destabilizing 1-PS, namely, X r (t) •M'has a limit as t — > 0, but 
( Xr , Xr) = - 1 < 0. Consider the case (b). If Z = O, X r for r = (0, 0, 0, 0, - 1 , - 1 ) 

is destabilizing. If rankZ = 1, we may assume Z = I 1 „ J taking suitable 

bases for V and J2y. AZ = BZ = O implies an = bn = 0. Then, A r for r = 
(0,0,0,0, —1,0) is destabilizing. This proves that *P is surjective if *¥ is (Gy,Xy)- 
semistable. 

(2.3.4) Now we assume that dim^W <g> S ) < 2 for W = C • ( J . This is 
equivalent to say that 

(a) Z12 = Z22 = and one of X2 or V2 is zero, or 
<■»*=» = «. fe)^.-dA (-) = , (-) =0. 

In the case (a), say, .£2 = 212 = 222 = 0, A r for r = (0, 1 , 0, 1 , 0, 0) is destabilizing. 
In the case (b), we may assume ( Zn J = (9) . Then, A ( Zl2 J = B ( Zl A = 
implies an = bn = 0. Therefore, A r for r = (0, 1,0,0,0, 1) is destabilizing. 

(2.3.5) Assume that dim^W ® @ s ) = 2 for W — C • ( ) , i.e., assume either 

(a) zn = 222 = 0, x 2 ^ 0, and y 2 ^ 0, or 

(b) Only one of x 2 or y 2 is zero, ( Zl2 j ^ 0, and A ( Zl2 ] = 5 ( Zl2 j = 0, or 

(c)x 2 =y 2 = 0, ( Zn ) ^0, and at least one of A ( Zl2 | orflf Zl2 ) is not zero. 

In the case (a), A r for r = (0, 1, 1, 1,0,0) is "strictly destabilizing", i.e., X r (t) -W 
has a limit as t — > 0, but {Xy,^r} =0^0. Similarly, in the case (c), A r for 
r= (0,1,0,0, 1,1) is strictly destabilizing. In the case (b), we may assume^ = 

and ( ) = I 1 ) • Then, as before, we have an = bn = and A r for r = 

\z22j Vv 

(0, 1,0, 1,0, 1) is strictly destabilizing. This finishes the proof of (iii)=4>(ii). 

(2.3.6) Now we prove the converse (ii)=^(iii). Suppose that *P is not (Gy,Xy)- 
semistable (resp. stable). Then, up to a change of bases for V and =2 r , there 
exists some non-zero r G Z 6 such that lim t ^oXr(t) • ^ exists but (Xy,k r ) < 
(resp. ^ 0). Let us call this r a destabilizing vector. Note that we may assume 

= n ^ r 2 , r 3 < r 4 , r 5 ^ r 6 (3) 



NON-LOCALLY-FREE LOCUS OF O'GRADY'S TEN DIMENSIONAL EXAMPLE 9 

by symmetry. Under this condition, (# y , A r ) < (resp. ^ 0) is equivalent to 

2r 2 > (resp. » r 3 + r 4 + r 5 + r 6 . (4) 

(2.3.7) Assume that r with r^ < is a destabilizing vector for X V. Then, we must 
have x\=X2 = 0, and therefore *P cannot be surjective. Assume that r with r$ < 
is a destabilizing vector for V I / . Then we have z\\ = zn = 0. Furthermore, if we 
even have rg < 0, zn and zn must vanish, therefore *¥ cannot be surjective. So, 
let us assume rg ^ 0. Then we automatically have rg > r$, so that a\2 = bn = 0. 
Noting that A 2 = O = B 2 , we have an = CI22 — b\\ — ^22 = 0- Thus, we must 
have AZ = BZ = O and *F cannot be surjective. 

(2.3.8) Now let us assume *P to be surjective. This implies that rj, r 4 , rs, rg ^ 0. 
For a destabilizing vector r, we deduce T2 > (resp. ^) from ©. If r3, r4, r$, rg < 
r2 for a destabilizing r, we get %2 = y>2 = zn = Z22 = and dim l I / (W £§> ^5) = 
< 2 for W = C • ( ° 

(2.3.9) Suppose r destabilizing such that exactly three of r3,r4,r5,rg are less 
than T2, i.e., r3,r4,r5 < r2 or r3,rs,rg < r2. In the first case, we have X2 = 
yi = 212 = 0- Moreover 012 = £12 = 0, since we must have r$ < rg. There- 
fore, dim v I'(VV r <g> &s) = 1 < 2 for W = C • I 1 ) . Also in the second case, we 

must have X2 = zn = Z22 = 0, which implies that dim v I / (VK <g) &$) = 1 < 2 for 
N 



W = £ - ' 1 

(2.3. 10) Now we assume that r destabilizing with exactly two of r?, , r$, r$ , rg are 
less than r2- Then we must have, say, r$, r 4 ^ r2- But under the strict inequality 
©, we get 

2r 2 > r 3 + r 4 + r 5 + r 6 ^ 2r 2 + r 5 + r 6 , 

which is contradiction since rs,rg ^ 0. This proves (ii)=Kiii) in the semistable 
case. 

(2.3.11) Finally, assume that *F is strictly (Gy,%y) -semistable. In this case, we 
can deduce from the argument in the previous paragraph that a "strictly desta- 
bilizing" vector r should satisfy that two of r3,r4,rs,rg are equal to r2 and the 
others are 0, namely 



(a) r 3 = r 4 = r2 and r 5 = r 6 = 0, or 

(b) r 4 = rg = V2 and r?, = r$ = 0, or 

(c) r 5 =r 6 = r 2 and r 3 = r 4 = 0. 



10 
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It is immediate that we have the corresponding statement in (2.3.5) in each case, 
and therefore dim l P(W r ® &$) — 2 for W — C • I 1 ] . This completes the proof of 



Proposition [23] 



Q.E.D. 



3. Calculation of the invariant rings 
In this section, we actually calculate the homogeneous ring of semi-invariants 

oo 

%y = @A{Y 7 ) G ^y (5) 

n=0 



appeared in Theorem [2T21 for each partition type r\ and finish the proof of Theo- 
rem [TTTJ 



3.1. First, we treat the case where 7 is of type [l 4 ] . We include this case because 

this is the toy example for the calculation of all the remaining cases. In this case, 

4 
y y ^J]Hom(V,C), 



7=1 



n*\4 



G y = GL(V) x (C* 

Xy = (det v y 2 ■ id<c* • id c * • id c * • id c * , 

and the space A{Yy) Gy ^ consists of SL(V) -invariants whose character is %y- If 
we write *P G Yy as 

( X\ x 2 \ 

y\ yi 

Zl Z2 
\w\ W 2 J 

using a coordinate, the ring of SL(V) -invariants is generated by the Pliicker co- 
ordinates, namely, the (2 x 2)-subdeterminants 



Pn 



*¥ 



X\ X2 

y\ yi 


, P\3 = 


X\ X2 
Zl Z2 


5 


Pu = 


X\ X2 
W\ W2 


) 














P23 = 




y\ yi 

Zl Z2 


, P24 = 


y\ 


yi 

w 2 


, P34 = 


Zl 


Z2 

w 2 



subject to the Pliicker relation 

P12P34- P13P24+ P14P23 =0, 

which is the only relation between the Pliicker coordinates. If we calculate the 
weights for these SL(V) -invariants, we get 
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dety 


c* 


c* 


c* 


c* 


P12 


-1 


1 


1 








P13 


-1 


1 





1 





Pu 


-1 


1 








1 


P23 


-1 





1 


1 





P24 


-1 





1 





1 


P34 


-1 








1 


1 


ly 


-2 


1 


1 


1 


1 



Therefore, the space A(Yy) G v^ is spanned by 

U0= P12P34, Hl=Pl3P24: U2 = Pl4P23 

and we have a description of the homogenous ring "Ry Q as 

oo 

Ry = ®A{Y Y ) G y^ n y=C[uQ 1 u ll u 2 ]/{uQ-u l +U2). 

w=0 

Applying Theorem I2.2L this leads to the conclusion By = Proj Jly = P 1 . More- 
over, a point \P is strictly semistable if and only if one of the Pliicker coordinates 
vanishes (Proposition l2.3l) . Therefore, we get 

(5 r flE),. e ^ = (uq — ui + u 2 = 0, uqu\U2 = 0) = { 3 points } C P 2 . 



3.2. Let ybe of type [1 2 ,2] and W = C 2 . Then, 

Y y = Hom(V, C) x Hom(V, C) x Hom(V, W) x N 2 , 

G r = GL(V) xCxCx GL(W), 
Xy=(det v y 2 -id c *-id c *-(det w ). 

Just as in §3.1, first we calculate the invariant ring A(Yy) SL ( v > xSL ( w \ and then 
pick up the homogeneous subring of elements with characters %'L (n ^ 0). Let 
us write *P = (iff, a) e Y y = Hom(V,r(i? y )) x N r as 

( x\ x 2 \ 

yi n_ 

Zll 212 
\Z2\ Z22) 

using a coordinate. As GL(V) acts trivially on N 2 , the generators for SL(V)- 
invariant ring A(Yy) SL ( v > is always given by Pliicker for the coordinates of \]/, 
plus the entries of A and B. Let 



*¥ 



¥ 



a = (A,B)), ((A,B)eN 2 ) 



/1 



x\ x 2 

y\ y2 



h 



Z\\ Z\% 

z 2 \ z 22 



W\ 



X\ x 2 

Zll Z12 

X\ x 2 

Z 2 l z 22 



w 2 



y\ y2 

Zll Zl2 

y\ y2 

Z2l Z22 



\ 
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SL(W) acts trivially on f\ and fi, and by left multiplication on w\ and w>2. Here, 
the situation came out is exactly the problem of invariants for mixed tensors. 
Classical invariant theory had a method to deal with this kind of problem, the 
symbolic method ([PV94], Theorem 9.3). The statement relevant to the current 
situation is the following: 

Let W be a vector space and U = W® 2 ©End(W)® 2 . Let SL(W) 
act on U by left multiplication on the factors W and by conju- 
gation on the factors End(V7). Represent an element of Y by 
(wi,w 2 ;A,B). Then, the ring of SL(W) invariants A(U) SL W is 
generated by 

• tr(VK (A, J B)),and 

• det(Wi(A,B)w< | W 2 (A,B)wj) (1,7 e {1,2}), 

where Wt(A, B) (i = 0, 1 , 2) stands for an arbitrary word in A and 
B. 

This claim is not strong enough in general, because it gives only infinite set of 
generators. The symbolic method say nothing about a bound on the generating 
set, whereas it should be actually finite by the famous theorem of Hilbert. How- 
ever, in our situation, the endomorphisms A and B are nilpotent and commute 
each other, so that the set of non-zero words in A and B is effectively bounded, 
and so is the generating set of invariants. More concretely, every SL(V) x SL(W)- 
invariants in our case is a polynomial in 

£1 =/lj £2 =J2, 



det(Avt>i I wi) 
det(Awi I W2) 
det(Aw2 I VV2) 



det(5wi I wi) 
det(5wi I W2) 
det(5w2 I W2) 



(Note that det(wi | W2) = f\ ■ fi). The weights of these invariants with respect to 
the characters are given by the following table. 





detv 


C*(y) 


C*(z) 


detiy 


§1 


-1 










^2 


-1 


1 


1 





^ 


-2 


2 







^ 


-2 


2 







^5 


-2 


1 


1 




^6 


-2 


1 


1 




^7 


-2 





2 




& 


-2 





2 




Xr 


-2 


1 


1 
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Therefore, the homogeneous invariant ring Ry = (& n >oA(Yy) Gr,x Y is generated 
by the elements of degree 1 

and the elements of degree 2 

&Hi <^8; ^4^7, <^8- 

Now we have to decide the relations between these generators. As we have a 
ring homomorphism 

C[&,...,&] ^A(Y Y ) SL ^ SL ^ ^A(Y r ), 

the kernel of this morphism is the module of relations between <fj;'s. As we are in 
a completely concrete situation, we can calculate this ideal using the elimination 
property of Grobner basis implemented on computer algebra system. The author 
used Singular HGPS09II to calculate the ideal. The result is that the relations 
are generated by 

&&-&&, &§7-&&, (6) 

This immediately implies that we have an isomorphism of homogeneous rings 

,Ui,U 2 ], 



n? c^ 



therefore, 5 r £ P 2 



For every point x V=(\j/, (A, B)), we have a point in the GL(W)-orbit of *P such 

that 

'0 0\ A) 0^ 

for A and B commute. At such a point, uq, u\, and u 2 can be written as 



UQ 



U\ 



XI 


X2 




211 


212 




y\ yi 




221 222 


•) 


CI ' 


X\ x 2 




y\ yi 




211 


Z 


12 




211 


212 



u 2 = b- 



X\ x 2 
211 212 



yi yi 

211 212 



It immediately follows from this expression that the points in the orbit corre- 
spond to a strictly semistable sheaf exactly if uq = or u\ = u 2 = (cf. Propo- 
sition [23] (ii)). Therefore, 

(B r nZ) red = (wo = 0) U (m =u 2 = 0)c F 2 [uq : u\ : u 2 ], 

which is a disjoint union of a line and a point on P 2 . 
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3.3. Let 7 be of type [2 2 ] and W\ = C 2 , W 2 = C 2 . Our GIT setting in this case 

is 

Y y = Hom(V, Wi) x Hom(V, W 2 ) xiV 2 x N 2 , 

G y = GL(V) x GL(Wi) x GL(W 2 ), 

i-2 



X Y = (dety) 7 ■ detivj • de% 2 . 



Represent *P e Fy as 



¥ 



¥ 



(x\\ x n \ 

x 2 \ y 22 



,a=(Ai,Bi;A 2 ,B 2 ) 



yn yi2 

Vj21 J22/ 

by a coordinate. Then, 5L(V) -invariants are generated by the entries of A\, B\, 
A 2 , B 2 , and 



f\ 



X 2 \ X 22 



h 



y\\ yn 

yi\ yn 



W\ 



( 



x\\ x\ 2 

yn yn 

x 2 \ x 22 

y\\ zn 



w 2 



( 



X\\ X\ 2 

yi\ yn 

x 2 \ x 22 

yi\ z 22 



SL(W\) acts trivially on fi,f 2 ,A 2 ,B 2 , by left multiplication on W{,w 2 , and by 

adjoint on A\,B\. Therefore, every SL(V) x SL( Q) -invariant is a polynomial of 

£1 = /l, £2 = fl, the entries of A 2 , B 2 , and 

<^ 3 = det(Aiwi I w\), £,4 = det(Biwi | wi), 
§5 = det(Aiwi I w 2 ), <^ 6 = det(#iwi | w 2 ), 
^7 = det(AiW2 I w 2 ), !;% = det(5iW2 | w 2 ). 

It is straightforward to check that GL(W 2 ) acts on the vectors 





via GL(Sym 2 W 2 ) . Furthermore, we have an isomorphism of s[(W2)-representations 

T:Sym 2 W 2 ^sl(W 2 ), 




--:X h 




X 2 . 



Therefore, a generating set of SL(V) x SL(W\) x SL(W 2 ) -invariants is given by 
the matrix invariants for A 2 , B 2 ,X\, X 2 , together with f\ and f 2 . The symbolic 
method tells us that the generating set of matrix invariants are given by traces of 
words in A2,52 5 ^i 5 an d X 2 . Note the following properties: 
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• We can sort the word in trace, i.e., tr(ZF) = tr(FZ). 

• We have A\ = A 2 B 2 = B\ = 0. 

• Xf = (<^ 5 2 — §3 £7) I 2 = O (see the relations ©). Similarly, we have 

x x x 2 = xl = o. 

Hence, the ring of SL(V) x SL(W{) x SL(W2) -invariants is generated by 

Ci = £1 = /lj £2 = %2 = hi 

C 3 = tr(A 2 X l ) , C4 = tr(B 2 Xi ) , £ 5 = tr(A 2 X 2 ) , £ 6 = tr(B 2 X 2 ) , 
and the table of the weights is as the following. 





dety 


det^ 


detiv 2 


Ci 


-1 







C2 


-1 







C3 


-2 






6 


-2 






c 5 


-2 






C6 


-2 






*r 


-2 







We have an obvious relation £4^5 — £3^ = and actually one can easily check 
that this is the only relation between £/'s. Summarizing, we get an isomorphism 
of graded rings 



"Jly = C[uq,Ui,U2,U3,U4]/(uiU4 — U 2 U 3 ) 



where 



u — Cl C27 "1 — ?3 5 M 2 — £45 "3 — C5) U A — &• 

Namely, 5y is isomorphic to a quadric cone in P 4 . 
At a point ¥ = (\jf, (Ai,Bi;A 2 ,B 2 )) e Y y such that 







«0, 



v «i 
, U4 are written as 



B\ 



UQ 



x n X12 

X21 *22 




fci 



yn yi2 

J21 J22 



A 2 




a 2 



5 2 




Z> 2 



*? 

u\ = a\a 2 D , u 2 = a\b 2 D , U3 = b\a 2 D , W4 = b\b 2 D , 



where D 



. Noting that 



xn X12 
M i = M 2 = "3 = M4 = <=> D = 0, or a\ = b\ = 0, or a 2 = b 2 = 0, 



it is easy to see (cf. Proposition [23l (ii)) that 

(ByniZ) rec i = (uq = 0) U (wi = u 2 = 113 = U4 = 0) C By C P [wo : ■ ■ -u\\ 
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namely, (fiyflL)^ is a disjoint union of a smooth hyperplane section and the 
vertex of By. 



3.4. Let 7 be of type [1,3] and W = C 3 . Then, we have 
Y y £ Hom(V, C) x Hom(V, W) x N 3 , 



G r = GL(V)xC*xGL(W), 
£ r =(det v ;r 2 -idc*-detiv. 



Represent *F G 7 r as 



¥ 



T 



yn yn 



a = (A,B] 



X\ x 2 

yn yn 


, fl = 


X{ x 2 

yn yn 


> h = 


X{ x 2 
3 ; 31 3 ; 32 


yn yn 

yn yn 


, 82 = 


yn yn 

3 ; 31 3 ; 32 


, g3- 


yn yn 

3 ; 31 3 ; 32 



yn yn 

\3 ; 31 3 7 32/ 

using a coordinate. The ring of SL(V) -invariants is generated by the entries of A 
and B, and the Pliicker coordinates 

f\ 

8\ 
If we put them into a vector and an alternating form 

//A 

w- I/2 , <0 = gi(ei Ae 2 )+g2(>i Ae 3 ) +g3(<?2 Ae 3 ), 

w 

it is easy to verify that SL(W) acts on the first vector by left multiplication and 
on the second form via GL(A 2 W) by the induced representation. We have an 
isomorphism of GL{W) representations 

T : A 2 W -> W w = Hom(W, C), 

under which 

T(co) = (g 3 -g 2 gi) =: A. 

According to the symbolic method, the ring of mixed SL(W) -invariants of a vec- 
tor, a co- vector, and matrices is given by 

(a) XWq(A,B)w 

(b) tr(W (A,B)) 

/ AWi(A,B) 

(c) det(Wi(A,B)w | W 2 (A,B)w | W 3 (A,fl)w) and det AW 2 (A,£) 

\AW 3 (A,B) 
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where w eW, X EW W , A, 5 e End(W), and W t (A,B) (i = 0, 1,2,3) stands for 
any word in A and B. Note that we have the commutativity and the nilpotency 
conditions 

AB-BA = <9, A 3 =A 2 B=AB 2 =B 3 = O. 

It is immediate that the invariants given by traces of words in A and B as in (b) 
vanish. For the type (a), the non-trivial invariants are 

£i = XAw, & = XBw, 

& = XA 2 w, %4 = XABw, % 5 = XB 2 w 

(note that X w = 0, which is nothing but the Pliicker relation). For the type (c), we 
have quite a few candidates. Notethatdet(Wi(A,5)w | W 2 (A,5)w | Ws(A,B)w) = 
if all of Wi(A,B) are non-trivial, because A and B are nilpotent and commute 
each other. Similarly, a rank consideration of the endomorphisms A'B J shows 
that the determinants of type (c) other than 
Vi = det(w \Aw | Bw), 

X>2 = det(w | Aw | A 2 w) , U3 = det(w | Aw \ ABw) , V4 = det(w | Aw \ B 2 w) , 
V5 = det(w I Bw I A 2 w) , V(, = det(w \ Bw \ ABw) , Vj = det(w | Aw | 5 2 w) 

must vanish. We also have seven (possibly) non-trivial invariants £1, . . . , £7 of 
type (c) involving A in the same way. These 19 invariants in total generate the 
invariant ring A(Yy) SL ( v ) xSL ( w \ The table of the weights of the invariants is the 
following. 





dety 


id c * 


detiv 




6 

Si 


-2 
-3 
-3 


1 
1 

2 


1 (i=l,.- 
3 (i=l,.- 
(i=l,.- 


•,5) 
•J) 

•,v) 


*y 


-2 


1 


1 





Therefore, the homogeneous invariant ring Jiy is generated by 
^(/=l,---5) and v r Q (ij = 1,- •• ,7). 

Now we look at the relations among them. A Grobner basis calculation tells 
us that all the invariants of the form u, • £j are contained in the homogeneous 
subring generated by £1 , • • • , £5 and the only remaining relation is rather obvious 

^-^5=0. 

This is equivalent to say that we have an isomorphism of homogenous rings 



% = 



J!?"* ) £5] /(<*!. -fefey 



In other words, By is isomorphic to a rank 2 quadric in P 4 . 
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Claim. A semistable point *P G Yy corresponds to a strictly semistable sheaf if 
and only if £3 = ^4 = £5 = 

Proof. Passing to a point in the GL(W) -orbit, we may assume that *¥= (\j/,(A,B)) G 
Yy is of the form 

N 
b\ 

^2 b 3 0, 




5: 

If we evaluate £3, £4, | 5 at this point, we get 
£3 = «l«3./igl =aia 3 

£,4 = a\b 3 f\g\ =a\bs 
& =bifafigi =b{b 3 



X\ 


*2 




yn yi2 


yu 


yi2 




J21 J22 


X\ 


X2 




yn yi2 


yu 


yi2 




J21 J22 



Xl X2 

yn yi2 



yn yi2 

y2i y22 



*((c(J)). 



According to Proposition 12.31 we have to show that there exists a vector v G V 
such that dim l P((Cv) £g> 0$) = 2. This is obvious if one of the two (2 x 2)- 
determinants vanishes. 

The remaining case is that ci\a 3 = a\b 3 =b\b 3 = 0, but none of the determi- 
nants vanish. This condition is equivalent to 

ai=b\=Q or a 3 = b 3 = 0, 

noting that a\b 3 — a 3 b\, for AB — BA = O. We may assume X2 = and yn — 
by a base change on V. Assume a\ =b\ = 0. Then, 

(yn\ M 

C 3/22 ffiC Cf. 

On the other hand, if a 3 = b 3 = 0, we have 

(° 

= C©C y 2 i 1 , 

where the first C is the 1 -dimensional x-coordinate space. This concludes that 
£3 = (§4 = £5 = =>• semistable. The converse is similar and even easier. Q.E.D. 

3.5. Let us consider the last case, i.e., the case where 7 is of type [4]. Let 

W = C 4 . We have 

Y Y = Hom(V,W)xN 4 , 

G r = GL(V)x xGL(W), 
X 7 =(det v y 2 -det w . 



*P((C 
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Write ¥ e Y y as 



¥ 



V : 



All X\2\ 

X2\ xn 

*31 *32 

\x 4 l *42/ 



a = (A,B)) 



using a coordinate. As before, SL(V)-invarinants are generated by the Pliicker 
coordinates and the entries of A and B, and SL(W) acts on the Pliicker coordinates 



Pij 



Xi\ X i2 

Xjl X j2 



:k*<7<4) 



via induced action on A 2 W. Note that A 2 W = C 6 admits a non-degenerate sym- 
metric form 

(.,.) = A : A 2 W x A 2 W ->■ A% 2 C 
and we have a Lie algebra isomorphism 

r:s[(W)4so(A 2 W). 



Therefore, we can translate the problem on SL(W) -invariants into a problem on 
50 (A 2 W) -invariants. Namely, we need to know the classical invariant theory on 
50(6)-invariants. Again, the symbolic method (see [PV94] Theorem 9.3 and the 
table on p. 254) tells us the following: 

Let U = C 2m with a non-degenerate symmetric form (-,•}. Then, 
the ring ofSO(U) -invariants in C[£/©End([/) efc ] is generated by 

(a) (u,Wo(Xi,---,X k )u),md 

(b)det(Wi(Xi,...,Xt)»,...W 2m (Xi,...,jq fc )»), 
where (u, (X h - ■ ■ ,X k )) G U © End(U)®\ W t (i = 0,- -• ,2m) is 
any word in Xi , • • • : X k . 

Here, we have to be cautious about the fact that the isomorphism T preserves the 
Lie bracket, but does not respect matrix products . Namely, we have to include 
every possible non-zero words in T(A l B J ) (z, j ^ and i + j < 4), not only in 
T(A) and T(B), to get a complete generating set of invariants. But, we also note 
that we still have a reasonably bounded set of non-zero words, because all of 
r(A'5 J ) are nilpotent and commute each other. 
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Define u = ^ Piji^i A ej) G A 2 W and 
i<j<i<6 

§1 = (uJ(A) 2 u), & = (u,T(A)T(B)u), & = {uJ(B) 2 u), 

& = (u,T(A) 4 u), ^ 5 = (u,T(A) 3 T(B)u), ^ = (u,T(A) 2 T(B) 2 u), 

^ = (u,T(A)T(B) 3 u), ^ = {uJ{B)\), 

^9 = (u,T(A 2 )T(B)u), &0 = (u,T(A)T(B 2 )u). 

One can check using a computer algebra system that the other non-zero invariants 
of type (a) and (b) above can be written as a polynomial of these £i , • • • , £,\q. This 
means that the invariant ring A(Yy) SL ^> xSL ^ w > is generated by <^-'s. Moreover, 
the weight of £,■ is always the same as the weight of %y, so that the homogenous 
invariant ring Jiy is generated by the degree one part Ift^i = C(£i, • • • , £io). 

Now we examine the relations among ^,'s. According to a Grobner basis 
calculation, again, the relations are given by 

rankl^ 4 ^ ^ f & )^ 1, (7) 

^6-2^7 + ^8-^0 = 0, 

&&-2&& + ^7 + ^10 = 0, (8) 

^4-2^5 + ^1^6-^ = 0. 

The determinantal equation © is nothing but the defining equation of the sin- 
gular rational scroll F = P>i (<ff® 3 © &{A) © ^(1)) C P 9 . The remaining three 
equations © defines a subvariety of this scroll, which is isomorphic to By. Take 
the natural proper birational morphism 

jl : F = P P i (^® 3 © ^(4) © ff(l)) -> F 

and consider the strict transform B' = [i~ l By. In terms of the bihomogeneous 
coordinate [t\ , ti\ C,\ , • • • , £5) on F, \x is described by 

jU §4 =^£4, jU 55=^^45 M S6=^1^2b4, M S7=^1^2b4, M S8 = ^2<=4, 
/i*&=*lC5, /X*^10 = -feC5- 

Pulling back the equations ® by these relations, we know that B' y is a zero locus 
of a single equation 

(f 2 2 Ci -2fi? 2 C2 + ^C 3 ) ■ C4- C 5 2 = 0. (9) 

This implies BL restricted to a fiber P 4 of F — > P 1 is a rank 2 quadric. In particular, 
By is a prime divisor in the singular scroll F. The image of the family of the 
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vertex lines of the quadrics © is contained in the plane II = P 2 [<^i : £2 : £3] and 
defined by 

t\Z, x -2tit 2 Z 2 + tfo = ([ti : fe] G P 1 ). 

The envelope A of this family of lines on II is a smooth conic defined by 

Claim. (B r fl E) re j = II, i.e., *P G iy determines a strictly semistable sheaf if and 
only if £4 = • • • = £10 = 0. 



Proof. We show that there exists a vector v G V such that dim v I / ((Cv) <g> £?$) = 
(Proposition [23] (ii)) if *P is a semistable point satisfying £4 = • • • = ^10 = 0. 



5 



/0 0\ 

fci 

b 2 b 3 

\&4 £5 ^6 0/ 



As before, we may assume that *¥ = (\j/, (A,B)) G Yy satisfies 

/o o o\ 

ai 

a 2 a 3 
y a4 «5 a6 0y 

If we evaluate the functions £4, ..., £10 at such a point, we get 

£ 4 = 2a 1 a 2 ^ • Pn , £5 = 2a 1 a 3 a 6 b 3 ■ p\ 2 , 
<^ 6 = 2{a\a 3 b 3 b (i + a 3 a 6 bib 3 ) ■ p\ 2l 
£7 = 2a 3 bib 3 b 6 ■ p\ 2 , | 8 = 2bib\b b ■ p\ 2 , 

E, 9 = {a 5 b\b 3 - a 3 b\b 5 - a 3 b 2 b 6 + a 2 b 3 b 6 )p\ 2 + (a 6 bib 3 - a\b 3 b£)p\ 2 p\ 3 
€10 = (a 3 a 6 b 2 - aia 5 b 3 - a 2 a 6 b 3 + aia 3 b 5 )p\ 2 + (a { a 3 b 6 - a 3 a 6 b 1 )p n p l3 



Ifpi2 



x n *i2 
■^21 *22 



0, we may assume #12 = ^22 = 0. Then, 



¥((C 







C 



M 


1 

w 



iC 



M 



W 



So, let us assume P12 7^ 0. We may assume 



xn *12 

x 2 \ x 22 



1 
1 



by a coordi- 



nate change. Note that AB — BA = O implies 

ai\a 3 :a 6 = bi\b 3 : b 6 . 
Thus, ^4 = • • • = (§8 = implies 

a\ = b\ = or a3 = b 3 = or a6 = b(, = 0. 
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First, we assume #3 = £3 = 0. In this case, we automatically have ^9 = £10 = 0. 
Then, 

/ \ M 



¥((C 



c 



1 

\X32j 







W 



Next, we assume a^ = b(, = 0. Then AB — BA implies a^ : as = £3 : b$, i.e. 
two vectors 

and 




03 

W 





£3 



are proportional to each other, and t, 9 = | 10 = is again automatic. By the 
semistability of *¥, one of these two vectors, say f (0,0, 6(3, as), does not vanish. 
Then, 

°' u g>^ 5 )=C l ©C ° 

X 22 «3 

\*32/ \«5/ 

Finally, assume ai = b\ = 0. In this case, we have a 2 : «3 : ^6 = ^2 : ^3 : ^6> so 
that | 9 = £ 10 = 0. Then, 

M 



^((c; 



Aty/ 



a 3 
-a 2 







w 



5i// 



03 

-«2 



Therefore, 



¥((C 



a 3 
-a 2 



C 



\ 



a 3 

-a 2 

&3 X 2\ — 0-2*22 

\a3X3{-a 2 X3 2 J 



)C 



M 




w 



Q.E.D. 



Claim. (5y n E 1 ) re j = A. 



Proof. Thanks to the previous claim, we can restrict ourselves to the locus of 
*¥ such that Ker^ = I Zl ®h 2 , where Z ( - e Hilb 2 (5), Supp(Z) = {p} (7= 4/?). 
Namely, take *P as 

/l 0\ /0 (A /0 0\ 



V 



V : 




1 



^ 



ai 
0000 

\0 a 6 0/ 



,5 



b\ 


\o ^ 6 0/ 



))■ 
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Note that KerM' is of the form I^ 2 exactly if a\ : b\ — a^ : b(,. If we evaluate t,\ , 
£,2, and <^ 3 at such a point, 

£1 = -2aia 6 , E, 2 = -a\h - a 6 bi, £ 3 = -2bib 6 . 

Therefore, a\ : b\ = a^ : b(, if and only if ^i ^3 — ^ = (note that we are consid- 



ering only the reduced structure). 



Q.E.D. 
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